Shaped-based IMU/Camera Tightly Coupled Object-level SLAM using Rao-Blackwellized Particle Filtering

Simultaneous Localization and Mapping (SLAM) is a decades-old problem. The classical solution to this problem utilizes entities such as feature points that cannot facilitate the interactions between a robot and its environment (e.g., grabbing objects). Recent advances in deep learning have paved the way to accurately detect objects in the image under various illumination conditions and occlusions. This led to the emergence of object-level solutions to the SLAM problem. Current object-level methods depend on an initial solution using classical approaches and assume that errors are Gaussian. This research develops a standalone solution to object-level SLAM that integrates the data from a monocular camera and an IMU (available in low-end devices) using Rao Blackwellized Particle Filter (RBPF). RBPF does not assume Gaussian distribution for the error; thus, it can handle a variety of scenarios (such as when a symmetrical object with pose ambiguities is encountered). The developed method utilizes shape instead of texture; therefore, texture-less objects can be incorporated into the solution. In the particle weighing process, a new method is developed that utilizes the Intersection over the Union (IoU) area of the observed and projected boundaries of the object that does not require point-to-point correspondence. Thus, it is not prone to false data correspondences. Landmark initialization is another important challenge for object-level SLAM. In the state-of-the-art delayed initialization, the trajectory estimation only relies on the motion model provided by IMU mechanization (during the initialization), leading to large errors. In this thesis, two novel undelayed initializations are developed. One relies only on a monocular camera and IMU, and the other utilizes an ultrasonic rangefinder as well. The developed object-level SLAM is tested using wheeled robots and handheld devices, and an error (in the position) of 4.1 to 13.1 cm (0.005 to 0.028 of the total path length) has been obtained through extensive experiments using only a single object. These experiments are conducted in different indoor environments under different conditions (e.g. illumination). Further, it is shown that undelayed initialization using an ultrasonic sensor can reduce the algorithm's runtime by half

Localization on Smartphones Using Visual Fingerprinting

Many people nowadays can benefit from localization since the ubiquitous smartphones are integrated with many facilities such as visual sensors which can provide absolute or relative localization. Visual Fingerprinting (VFP) is an image matching technique which is based on visual information and provides localization to the user by finding the closest images in the database of images or videos. This approach has the flexibility to compensate between the accuracy and the practical implementation requirements. VFP especially becomes an attractive localization solution for the urban indoor environment due to certain number of facts such as limited access to the GNSS signals, the abundance of visually recognizable and distinguishable features in these environments, and that it does not require device calibration. Most VFP methods that are developed to be implemented on the smartphones include integration with other sensors such as Wi-Fi. These methods are focused mostly on the accuracy and less so on the computational aspects. They usually do not provide a detailed analysis that would suggest the feasibility of real-time implementation of their methods on smartphones. This study investigates the effectiveness of different VFP methods by evaluating the accuracy of the matching, preprocessing time and average matching time. To perform VFP, several things are required; a database of the location-tagged images, an algorithm to process the uploaded images; an algorithm to find the match in the image database and a localization algorithm that infers a location for the user based on the aforementioned steps. This research is focused on the first three steps. Explicitly the local detectors and descriptors have been used and their important properties have been studied. To further enhance the performance, multi-dimensional data structure has been deliberated. Consequently, a set of different combination for detectors/descriptors, data structures and dimensionality reduction algorithm has been chosen to be evaluated. The results obtained from the evaluations show that with the proper selection between these combinations, they can deliver the practical requirements to implement VFP-based localization on smartphones